Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement upload_file endpoint #547

Merged
merged 3 commits into from
Jul 4, 2024
Merged

Implement upload_file endpoint #547

merged 3 commits into from
Jul 4, 2024

Conversation

ykeremy
Copy link
Contributor

@ykeremy ykeremy commented Jul 4, 2024 •
edited by ellipsis-dev bot
Loading

Uploads to:

s3://skyvern-uploads/<ENV>/<ORG_ID>/<TODAY'S DATE>/<UUID>_<FILENAME>
Ex:
s3://skyvern-uploads/local/o_221157791538210890/2024-07-03/2e95b6fd-9dd6-4c1b-a79f-0e8a24ab14c5_cat.avif
🚀 This description was created by Ellipsis for commit 726cdcc

Summary:

Added upload_file endpoint to upload files to S3, initialized aws_client, and included methods for file stream uploads and size validation.

Key points:

  • Added upload_file endpoint in skyvern/forge/sdk/routes/agent_protocol.py to upload files to S3.
  • Endpoint uploads file to s3://<bucket>/<env>/<org_id>/<date>/<uuid>_<filename>.
  • Returns S3 URI and presigned URL for the uploaded file.
  • Added aws_client initialization in skyvern/forge/sdk/api/aws.py.
  • Removed AsyncAWSClient initialization from skyvern/forge/app.py.
  • Handles file reading, S3 upload, and presigned URL generation.
  • Added aws_client.upload_file_stream method in skyvern/forge/sdk/api/aws.py to handle file stream uploads.
  • Added validate_file_size function in skyvern/forge/sdk/routes/agent_protocol.py to check file size before upload.
  • Updated skyvern/config.py to include MAX_UPLOAD_FILE_SIZE configuration.
  • Example S3 path: s3://skyvern-uploads/<ENV>/<ORG_ID>/<TODAY'S DATE>/<UUID>_<FILENAME>

Generated with ❤️ by ellipsis.dev

@ykeremy ykeremy added the sync label Jul 4, 2024
ellipsis-dev[bot]
ellipsis-dev bot reviewed Jul 4, 2024
View reviewed changes
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 42ba863 in 41 seconds

More details
  • Looked at 122 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_xLBVCtMoytuhnRVa

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

skyvern/forge/sdk/routes/agent_protocol.py Outdated Show resolved Hide resolved
ellipsis-dev[bot]
ellipsis-dev bot reviewed Jul 4, 2024
View reviewed changes
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 42ba863 in 56 seconds

More details
  • Looked at 121 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_D2ir5aicMD9gHwTo

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

skyvern/forge/sdk/routes/agent_protocol.py Outdated Show resolved Hide resolved
@ykeremy
🔄 synced local 'skyvern/' with remote 'skyvern/'
af269db 
Uploads to:
```
s3://skyvern-uploads/<ENV>/<ORG_ID>/<TODAY'S DATE>/<UUID>_<FILENAME>
Ex:
s3://skyvern-uploads/local/o_221157791538210890/2024-07-03/2e95b6fd-9dd6-4c1b-a79f-0e8a24ab14c5_cat.avif
```
<!-- ELLIPSIS_HIDDEN -->

----

| 🚀 | This description was created by [Ellipsis](https://www.ellipsis.dev) for commit 9fcc5363e207314ff99811721274db6b757aa4fc  |
|--------|--------|

### Summary:
Added `upload_file` endpoint to upload files to S3 and return presigned URLs, with `aws_client` initialization moved to `skyvern/forge/sdk/api/aws.py`.

**Key points**:
- Added `upload_file` endpoint in `skyvern/forge/sdk/routes/agent_protocol.py`.
- Endpoint uploads file to `s3://<bucket>/<env>/<org_id>/<date>/<uuid>_<filename>`.
- Returns S3 URI and presigned URL for the uploaded file.
- Added `aws_client` initialization in `skyvern/forge/sdk/api/aws.py`.
- Removed `AsyncAWSClient` initialization from `skyvern/forge/app.py`.
- Handles file reading, S3 upload, and presigned URL generation.

----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)

<!-- ELLIPSIS_HIDDEN -->
@ykeremy ykeremy force-pushed the ykeremy/file-upload-endpoint branch from 42ba863 to af269db Compare July 4, 2024 00:28
ellipsis-dev[bot]
ellipsis-dev bot reviewed Jul 4, 2024
View reviewed changes
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on af269db in 38 seconds

More details
  • Looked at 82 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_bVJM5jBNqPISnlYm

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

skyvern/forge/sdk/routes/agent_protocol.py Outdated Show resolved Hide resolved
@ykeremy
Copy link
Contributor Author

ykeremy commented Jul 4, 2024

TODO:

  • Stream into S3
  • Limit file size

ellipsis-dev[bot]
ellipsis-dev bot reviewed Jul 4, 2024
View reviewed changes
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 0f65662 in 39 seconds

More details
  • Looked at 81 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_Nv4L01v8oZmzJFnh

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

skyvern/config.py Outdated Show resolved Hide resolved
ellipsis-dev[bot]
ellipsis-dev bot reviewed Jul 4, 2024
View reviewed changes
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 726cdcc in 37 seconds

More details
  • Looked at 13 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_MXi2LCBhTUeCCEPo

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

skyvern/config.py Show resolved Hide resolved
@ykeremy ykeremy merged commit 21b9eea into main Jul 4, 2024
2 checks passed
@ykeremy ykeremy deleted the ykeremy/file-upload-endpoint branch July 4, 2024 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ellipsis-dev ellipsis-dev[bot] ellipsis-dev[bot] left review comments

Assignees
No one assigned
Labels
sync
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@ykeremy